First Steps toward Determining the Role of Visual Information in Music Communication

We reviewed the resuts of our previous experiments on comparing the emotions recognized by recipients for several types of stimuli. Until recently, we had assumed that visual information provided simultaneously with musical performance was useful in helping hearing-impaired people to recognize the emotions conveyed by music. After our review, we became unsure whether multichannel information that supposedly helps deaf and hard-of-hearing people to recognize emotions actually works well. In this paper, on the basis of a new question that we pose about multichannel music information, we describe an experiment on comparing stimuli provided by types of music only, by music with video sequences, and by video sequences only. The results show that visual information has less effectiveness for hearing-impaired people, though visual information does have a role in supporting the recognition of emotions conveyed by music.